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ABSTRACT 



Research has shown that student mid-term feedback has 



significantly increased subsequent ratings of teacher effectiveness, student 
achievement, and student attitudes when the feedback results were accompanied 
by expert consultation. A gap in the literature is an instrument intended to 
provide specific feedback on systematic planning and delivery of 
instructional activities that can enhance teaching along with student 
learning, and motivation. Specific feedback related to the Reiser and Dick 
Instructional Planning Model (1996) can fill this gap. Following a literature 
review, this paper proposes a study to examine the effect of mid-term student 
feedback and consultation related to the Reiser and Dick Model on teacher 
behaviors, student learning and motivation. The independent variable is 
mid-term student ratings feedback and consultation related to teacher 
utilization of the learning activities (2 levels) . The dependent measures 
will assess the impact of the intervention on teacher systematic instruction 
behaviors through observations, interviews and surveys; student learning as 
measured by final exam scores on a common exam; and student motivation 
through surveys. Based on teacher interviews and surveys, the study will also 
address the utility of the Reiser and Dick model for Teacher Assistants. 
Appendices include: a comparison chart of feedback studies; Reiser and Dick 
Instructional Activities link to Gagne and Learning Theory; overview of the 
problem statement (The Effect of Student Ratings of Systematic Instruction on 
Teaching, Learning, and Motivation) ; instructional activities feedback form 
(Florida State University Evaluation Services) ; consultation interview 
protocol; classroom observation checklist; Keller's Course Interest Survey 
(Florida State University Evaluation Services); and an overview of procedure. 
(Contains 72 references.) (AEF) 
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Abstract 

Research has shown that student mid-term feedback has significantly increased 
subsequent student ratings of teacher effectiveness, student achievement and student 
attitudes when the feedback results were accompanied by expert consultation. A gap in 
the literature is an instrument intended to provide specific feedback on systematic 
planning and delivery of instructional activities that can enhance teaching along with 
student learning and motivation. Specific feedback related to the Reiser and Dick 
Instructional Planning Model can fill this gap. 
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A review of literature on formative evaluation of teachers through mid-term 
student feedback and how the Reiser and Dick Instructional Planning Model can 
enhance this feedback. 

Ideally, teachers want to prepare and deliver the most effective lessons possible 
related to their course content so that students will effectively learn and also enjoy the 
learning process. There are numerous methods available to prepare teachers for this 
awesome responsibility such as formal pre-service teacher training, teaching workshops 
and assistantships, teaching manuals or simply ‘learning by doing.’ There Eire also 
numerous methods to assess the effectiveness of instruction by examining teaching 
behaviors, student performance and motivation in the classroom. Because of the 
increased number of students going to college and the increased research demands on 
university faculty, graduate student teaching assistants (TAs) teach many of the lower- 
level introductory courses. At a large Southeastern university, for example, TAs taught 
over 15,000 students across 39% of all the available courses offered in a Spring Semester 
(Frost, 1999). 

Unfortunately, approximately 50% of TAs do not receive any formal teacher 
training and the majority of TAs that do attend training participate in limited one day 
workshops before each term (Shannon, Twale, & Moore, 1998). The content for most of 
these workshops is also quite broad and often ignores detailed training on developing 
specific lesson plans. A survey of 18 teacher training manuals from prominent 
universities found that “none of the topics that dealt with planning individual class 
sessions reached the 50% criterion” (Lowman & Mathie, 1993, p.87). 
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One mechanism available to nearly all TAs is student ratings feedback on their 
teaching ability at the end of the course. Guidance on using feedback from these ratings 
was mentioned in 83% of the teacher training manuals listed above. Many researchers 
(Abbott et al., 1990; Cohen, 1980; McKeachie et al. 1980; Mertler, 1996; Overall & 
Marsh, 1979; Marsh & Roche, 1993; Wu, 1993) have investigated the influence of these 
student ratings used at mid-term on teacher effectiveness. Many studies have shown 
significant differences in teaching effectiveness, student learning and student affect for 
those teachers who receive mid-term student ratings feedback, especially when combined 
with consultation. 

The instruments used to provide feedback in such studies, however, are generally 
related to surveys of broad teaching constructs or behaviors that are found on typical end 
of semester student rating forms. McKeachie (1997) calls this instrument development 
process “dust bowl empiricism: that is, get a number of items about teaching and see 
what works.”(p. 1223). Thus, a noted gap in the literature is a student rating instrument 
developed based on learning theory. Marsh (1984) adds: “An alternative approach [to 
student feedback rating forms] based on a theory of teaching or learning could be used to 
posit the evaluation dimensions, though such an approach does not seem to have been 
used in student evaluation research” (p. 709). A more detailed review of the literature 
will illustrate major findings of the student feedback rating research and then introduce 
another proposed study that attempts to pursue recommendations of the prominent 
researchers in this important research topic. 
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Review of Literature 

History 

Gage et. al. (1963) and Tuckman and Oliver (1968) are two of the earliest studies 
examining the effect of student ratings feedback on teacher behavior or effectiveness. 
Gage et. al. (1963) found that 6 th grade teachers responded positively to student feedback 
and changed their behaviors toward student suggestions. Tuckman and Oliver’s (1968) 
study examined 286 high school vocational technical teachers and found that teachers 
receiving student feedback significantly changed teacher behaviors more than non 
feedback groups. Ironically, they also found the teachers to be more responsive to 
student feedback than supervisor feedback (principals) or student feedback combined 
with supervisor feedback. Since these early studies, Greenwald (1997) found 176 more 
studies on the topic. The interest peaked from 1976-1980 with 71 studies reported and 
gradually dropped in the 80s and 90s with only 8 studies reported from 1991 to 1995. 

Other researchers have extended the focus of student ratings beyond teaching 
effectiveness to student learning outcomes. Basically, if the students rate the teacher as 
more effective, then the students should learn more. Remmers, Martin, and Elliot (1949) 
published one of the first studies to examine this relationship of student ratings to 
learning with a multi-section design. Marsh (1984) describes the ideal conditions for such 
multi-section designs to enhance the validity of studies. These conditions basically 
require many sections of the same course teaching similar content and taking common 
exams. The overall learning correlation most frequently cited by meta-analyses of such 
studies is a moderate .4 to .5 (Cohen, 1981; d’Apollonia & Abrami, 1996, 1997). 

Cohen’s meta-analysis (1987) of 41 studies found the following relationships of feedback 
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ratings with student achievement: Structure, .55; interaction, .52; skill, .50; overall 
course, .49; overall instructor, .45; learning, .39; rapport, .32; evaluation, .3; feedback, 
.28; and interest/motivation, .15. 

After examining 25 years of research on this topic, Cashin (1995) and others 
agree that “In general, student ratings tend to be statistically reliable, valid, and relatively 
free from bias or the need for control: probably more so than any other data used for 
evaluation” (p. 6). Thus, there is a rich and worthwhile history of relevant research on 
this valuable topic. 

Feedback Studies 

Many studies have examined the impact of student ratings feedback on teaching, 
learning and affect in a variety of subjects, normally at the undergraduate level. Skeff 
(1983), however, also used feedback forms to examine the effect on the teaching 
effectiveness of doctors as rated by medical students. 

Cohen (1980), and L’Hommedieu, Menges, and Brinko (1990) conducted meta- 
analyses of the most valid studies, 17 and 28 respectively, on this topic and identified 
overall effect sizes and methodological issues. While comparing these two meta- 
analyses, the influence of consultation combined with midterm student ratings feedback 
becomes quite apparent. While the overall effect size is .342 in L’Hommedieu’s et. al 
(1990) analysis and .38 in Cohen’s (1980) analysis, both analyses revealed much larger 
effect sizes for studies that combined personal consultation with the written summary 
feedback. The average effect size for the consultation studies in L’Hommedieu et. al 
(1990) was .86 (1.032 by removing an outlier) and .20 for non-consultation studies. 
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Cohen (1980) found similar differences in consultation studies with effect sizes averaging 
.64 and .20 for non-consultation studies. 

Appendix A illustrates an expanded version of L’Hommedieu et al’s (1990) 
comparison chart of feedback studies. This chart, arranged in order of effect size, 
highlights the methodological procedures and variables examined in 24 studies, using 
effect sizes found by L’Hommedieu et al (1990, p. 234). Most studies examine the 
effects of the feedback on teacher effectiveness as measured by a dimension performance 
on the feedback form. Several studies, however, have also examined the impact of 
ratings feedback performance on affective variables such as attitude toward the course, 
subject, or instructor (Erickson & Sheehan, 1976; McKeachie, et. al, 1980; Overall & 
Marsh, 1979). 

Feedback Forms 

There are nearly as many feedback form instruments as there are studies in this 
field. For example, the 24 studies highlighted in Appendix A used 23 different types of 
student rating forms. The first documented student rating feedback scale was the Purdue 
Rating Scale of Instruction in 1929 (Darr, 1977). As mentioned earlier, most of these 
forms are a collection of items that the using institution developed, factor analyzed and 
believes represent effective teaching. 

Marsh’s (1982) Students’ Evaluation of Education Quality (SEEQ), which has a 
reliability of r = .9 with 25 person classes and higher reliability for larger classes, for 
example, has 9 dimensions of teaching. Erickson’s Teaching Analysis by Students 
(TABS) (1976) has three dimensions (stimulation, organization, and evaluation) with 
similar reliability coefficients. Several other common dimensions are skill, rapport, 
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structure, clarity, difficulty, interaction, enthusiasm, and feedback to students (Cohen, 
1980; Marsh & Roche, 1997; McKeachie, 1997). 

It appears that the more specific the items on the feedback form, the more likely 
the students will be able to identify and evaluate the teacher’s behavior. McKeachie 
(1994) states “...more specific items reporting perceptions or evaluations of teacher 
behaviors or specific aspects of the course are likely to be more helpful than very general 
items” (p. 328). 

Validity Concerns 

McKeachie (1997) also believes that “Student ratings are the single most valid 
source of data on teaching effectiveness” (p. 1219). Other researchers (Greenwald & 
Gillmore, 1997), however, question student ability to evaluate teachers and have 
examined several possible threats to the validity of such research. 

One hypothesis is that the grading leniency may impact student ratings. 

Basically, if the teacher gives high grades, then students will rate them higher. Other 
possible variables or ‘biases’ that have been examined are prior student differences; 
instructor prior knowledge of exams; teacher experience, autonomy or expressivity; 
course discipline and level; class size; and student motivation (d’Apollonia & Abrami, 
1997; Marsh, 1984; Marsh & Roche, 1997). 

Other threats investigated include the ‘John Henry’ effect, halo effect (Marsh & 
Roche, 1993; McKeachie, 1997), and the ‘Dr. Fox’ effect. The ‘Dr. Fox’ effect implies 
that enthusiastic and entertaining instructors can “seduce” students into higher ratings 
(Marsh & Roche, 1993, p. 1 193). Such an effect is normally captured by the feedback 
surveys that measure an ‘enthusiasm’ dimension. Marsh (1984) labels the attempt by 
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researchers to find bias in the feedback studies as a ‘witch hunt’ (p. 741). Overall, many 
researchers agree that bias in student ratings plays a very minor role (d’ Appolloni & 
Abrami, 97; Marsh, 1987; McKeachie, 1997; Murray, 1984) 

Recommendations for Future Research 

Researchers have offered several recommendations to possibly improve the 
findings and utility of future research in this topic. Several researchers who reviewed the 
literature (Abrami, d’Apollonia, & Cohen, 90; d’Apollonia & Abrami, 1997; Marsh, 

1 984; McKeachie, 1 997) suggest linking future feedback instruments to theories of 
cognition and motivation. Marsh and Roche (1997) also suggest linking student feedback 
ratings to more affective outcomes such as student motivation. Abrami, d’Apollonia, & 
Cohen (1990) suggest “increasing the quality and type of criterion measures used” ( p. 
230) for the outcome variable. Thus, using a feedback instrument related to learning 
theory and assessing its impact with a reliable instrument that assesses student motivation 
could contribute to a gap in the research in this worthwhile topic. 

A Proposed Study 

The following study attempts to explore several of the recommendations 
highlighted above and contribute to the gap in literature by using a feedback instrument 
related to learning theory. The study would also provide students with an opportunity to 
provide more specific feedback on specific teaching behaviors. Students would assess 
the teacher’s frequency of systematic instructional learning activities that they can 
observe in the classroom. Detailed consultation to explain the feedback and offer 
strategies to implement more systematic learning activities should enhance teaching 
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behaviors, student learning and student motivation. Measuring student motivation with 
Keller’s Course Interest Survey (1995) would also reveal more specific aspects of the 
impact on motivation. 

Research has shown that systematically designed instruction can produce greater 
learning outcomes (Bowsher, 1989; Morgan, 1989), however, there is a tendency for 
teachers to ignore these proven principles (Reiser, 1994). Young, Reiser, and Dick 
(1998) studied nine teachers that were rated ‘superior,’ the top 1% of 1,500 teachers in a 
school district, and discovered that these teachers were not incorporating systematic 
planning principles into their lesson plans. Reiser and Mory (1991) even found that a 
teacher trained in the systematic design of instruction used ‘sketchy’ written planning 
techniques but did undergo an effective ‘mental planning’ process. 

Gagne (1985) developed a systematic theory of instruction for learning based on 
his extensive study of the conditions of learning within the information processing 
learning theory (Gagne, Briggs, & Wager, 1992). His impact on the field of instructional 
design and technology is evident by his 131 citations in the literature from 1985-1990, 
which was the second most prominent in the field (Anglin and Towers, 1992). Reiser 
and Dick (1996) developed an instructional planning model for teachers that incorporates 
most of Gagne’s nine events of instruction (See Annex B). Klein (1991) has shown that 
pre-service teachers can quickly learn and apply these principles. These principles apply 
to learning in general and should not be restricted to specific disciplines. 

A critical component to Reiser and Dick’s systematic planning model is to 
"Revise instruction in light of student performance on each objective and student 
attitudes towards your instructional activities" (1996, p. 9). Driscoll (1994) echoes this 
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principle by stating “The best guide to planning instructional events, then, is the students 
themselves" (p. 357). The Reiser and Dick instructional activities provide a systematic 
and ‘teacher-friendly’ tool for TAs to help improve their planning and delivery of 
instruction, which should ultimately enhance learning. 

Statement of the Problem 

The purpose of this study is to examine the effect of mid-term student feedback 
and consultation related to the Reiser and Dick (1996) instructional activities model on 
teacher behaviors, student learning and motivation. The independent variable is midterm 
student ratings feedback and consultation related to teacher utilization of the learning 
activities (2 levels). The dependent measures will assess the impact of the intervention on 
teacher systematic instruction behaviors through observations, interviews and surveys; 
student learning as measured by final exam scores on a common exam; and finally, 
student motivation through surveys. Based on teacher interviews and surveys, the study 
will also assess the utility of the Reiser and Dick model for TAs. Appendix C provides a 
graphic overview of the problem statement. 

The research questions that I will investigate include: 

1 . Will midterm student feedback combined with consultation related to the 
Reiser and Dick instructional activities model: 

Change TA behaviors to increase use of systematic instructional activities in 

class? 

2 Improve student learning? 

3 Improve student motivation? And, 

4. Do teachers think the Reiser and Dick instructional activities model is 

beneficial to enhance their instruction? 
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METHOD 

Subjects 

The study will examine approximately 36 TAs and their undergraduate student 
section(s) at a Southeastern University during the Fall 2000 term with a pilot study of 4 
TAs in the preceding Spring term. The Departments of Computer Science and Chemistry 
have both committed to provide two TAs for the pilot study and approximately 1 8 TAs in 
the Fall for the lower level core classes on Computer Literacy and Basic Chemistry. 

These classes are solely taught and graded by the graduate (and some undergraduate) 
TAs. Each TA teaches between 18-24 students per section and normally has from 2 to 5 
sections of the same course. 

Shannon, Twale, and Hancock (1996) found that College of Education teachers 
use the greatest variety of teaching methods, while math and science teachers use the 
least. The Reiser and Dick model (1996) then, may be very helpful to the computer 
science and chemistry TAs in this study. 

Independent Variables 

Midterm feedback/consultation of student ratings on systematic instruction 

The two levels of this variable are no feedback on student ratings (level 1) and 
written feedback with personal consultation (level 2). The midterm student feedback is 
directly related to Teaching Assistant (TA) behaviors reflecting systematic instruction 
according to the Reiser and Dick (R & D) Model (1996). The ‘Instructional Activities 
Feedback Form’ for gathering this feedback is in Appendix D. Questions for this form 
were designed to assess the frequency of TA use of the six instructional activities 
outlined in the Reiser and Dick model as well as the two additional activities of 
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remediation and enrichment. There are one to four specific questions that assess each 
activity, which the student can readily identify in class. Question #3, for example asks if 
the TA “Begins lesson with an interesting or exciting fact, demonstration, or story related 
to the class topic.” 

The consultation interview protocol (see Appendix E) provides constructive 
feedback to the teacher based on the results of the midterm feedback form and classroom 
observations. Consultation procedures outlined by Brinko (1990), Cohen and Herr 
(1982), Erickson and Erickson (1979), and Wilson (1986) provided excellent guidelines 
for developing the consultation protocol to enhance implementation of the feedback. The 
protocol also attempts to assess the instructor’s willingness and feasibility to incorporate 
the instructional activities in future planning for classroom activities and how much the 
instructor values the feedback. 

Dependent Measures 

The dependent measures for this study will include teacher systematic 
instructional behaviors, student learning and motivation as well as a descriptive analysis 
of post intervention teacher beliefs about the utility of the Reiser and Dick model. 

Teacher Systematic Instructional Behaviors 

Teacher systematic instructional behaviors are defined as the teacher’s use of 
systematic procedures to develop and deliver objectives based instruction with activities 
resembling the Reiser and Dick instructional activities model. This variable will be 
assessed through student feedback, observations, and self-report to triangulate the 
assessment. 
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The Instructional Activities Feedback Form (Appendix D), which will be 
administered to all students near mid and end of term, has 26 questions that assess TA 
systematic teaching behaviors by rating the frequency of using the Reiser and Dick 
instructional activities. For example, question #1 1 states “[The teacher] uses several 
relevant examples to illustrate the lesson topic” and the student must respond to a Likert- 
type scale ranging from almost never to almost always (for a typical class). The initial 
reliability of this instrument will be conducted during the pilot study. 

The researcher will observe all TAs receiving feedback and selected TAs in the 
control group near the beginning and end of the semester using the Classroom 
Observation Checklist (Appendix F) to also assess TA behaviors. This qualitative data 
will descriptively summarize quantitative frequencies of behaviors in class and also 
provide more detailed descriptions of those behaviors. Murray (1983) found high 
correlations of behaviors in class with student ratings and concluded student ratings “can 
be accurately predicted from outside observer reports of specific classroom teaching 
behaviors” (1980, p. 31). Other researchers have also linked classroom observations to 
survey items (Land & Combs, 1981; Hines, Cruickshank, & Kennedy, 1982). 

Interviews with the TAs, using a consultation interview protocol (Appendix E) 
will also measure the impact of the intervention on TA behaviors (e.g. Do TAs report 
using more systematic behaviors in class). For example, a midterm consultation question 
(#6) asks “Do you believe that you can incorporate these activities into your classes?” 
After the intervention, the question will be changed to “Were you able to successfully 
incorporate these activities into your classroom? How often?” 
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Student Learning 

Student learning will be measured by computing each TAs average final exam 
score from a common course final exam (e.g. 81.3%). TAs will only be compared to 
other TAs in the same course (e.g. 18 in Computer Science, 18 in Chemistry). Overall 
and Marsh (1979) demonstrated significant differences in final exam scores between 
groups where all teachers administered the same exam for the same course. 

Student Motivation 

Student motivation in the classroom is defined as student interest in the class or 
subject content. All students will complete Keller’s (1995) Course Interest Survey (CIS) 
(Appendix G) near the middle and end of term to measure their interest in the course. 
This instrument assesses Keller’s (1987a, 1987b) ARCS motivation model components , 
(attention, relevance, confidence, satisfaction) in a course. For example, question # 7 
asks: ‘The instructor makes the subject matter of this course seem important.” The 
instrument has the following reliability ratings: attention: 0.84; relevance: 0.84; 
confidence: 0.81; satisfaction: 0.88; overall: 0.95. 

Beliefs on Utility of the Reiser and Dick Instructional Activities Model 

Utility is defined as teacher and student beliefs that the Reiser and Dick activities 
are worthwhile for the teacher’s academic course/discipline and the TA has an interest to 
use them in the future. The consultation interview protocol (Appendix D) administered 
to the experimental group has several questions that assess TA beliefs of the model’s 
utility, such as question #6, which asks “Do you believe that you can incorporate these 
activities into your classes?” Students also have an indirect opportunity to comment on 
the models utility by responding to question #30 on the Instructional Activities Feedback 
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Form (Appendix C), which asks “How would you rate the usefulness of these activities 
for promoting learning in this course?” 

Procedures 

Appendix H graphically summarizes the procedures for the proposed study. The 
36 volunteer TAs will be divided into stratified random samples to either the no feedback 
control group or the experimental feedback/consultation group. L’Hommedieu, Menges, 
and Brinko (1990) advise using stratified samples to enhance equivalent groups. 
Stratification will include teaching experience, subject and nationality. Wu (1993) found 
that TAs with the least experience benefited most from mid-term feedback. 

The researcher will observe all TAs in the experimental feedback/consultation 
group and randomly selected TAs in the control group during the first month of classes to 
qualitatively assess the instructional activities taking place in a typical class period. The 
Classroom Observation Checklist (Appendix F) enables the observer to record the 
frequency, examples, and suggested alternatives of the Reiser and Dick model 
instructional activities. 

Students in the TA classes from both groups will take the pretest in the 4 th or 5 th 
week of the term after the first graded assessment in the class. The pretest, which is 
actually an initial assessment of the TA and course, will consist of the Instructional 
Activities Feedback Form and the Course Interest Survey. Instructions for the surveys 
will follow standard student survey instructions with the students administering the 
surveys, collecting them, and depositing the sealed envelope of responses (with initials 
across the seal) into a pre-positioned confidential collection box which only the 
researcher has access. Surveys will be administered at the beginning of the class period 
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to reduce the student tendency to ‘rush’ through the form to depart the class quickly. 
Survey administration should last approximately 10-15 minutes. 

Immediately after data analysis, the researcher will personally contact all 18 TAs 
in the feedback group to schedule an interview to review the results of the Instructional 
Activities Feedback Form before the end of the 6th week of classes. This interview will 
follow the Consultation Interview Protocol (Appendix E) and help identify strengths and 
weaknesses from the feedback form and classroom observation and possible 
implementation strategies. The control group will not receive any midterm feedback on 
the surveys. 

The researcher will again observe all the TAs in the experimental group and 
selected TAs in the control group and record findings on the Classroom Observation 
Checklist. These observations will assess whether the TAs demonstrated more 
systematic teaching behaviors as a result of the feedback and consultation. 

In the last two weeks of the term, both groups will receive the posttest, which will 
mirror the pretest using both the Instructional Activities Feedback form and the CIS. 
Administration and tum-in procedures will be identical to the pretest and last 
approximately 10-15 minutes. Again, it is critical that surveys are administered at the 
beginning of a class period (not a test period) to enhance student attention. 

The final exam scores of the students will measure the impact of the 
intervention on student learning. All students take the same final exam for the same 
course curriculum in both courses. These exams are not cumulative and would only test 
the students on material they learned from the TA AFTER the midterm intervention. 

After final exams are graded, all TAs from both groups will submit their class exam 
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average by percentage (e.g. 83.4%) to the researcher for analysis. The researcher will 
also do a follow-up interview with the TAs to again assess the utility of the model and 
feedback for their future teaching. 

Design and Data Analysis 

This study combines experimental and qualitative methods to greater assess the 
impact of the instructional activities feedback and consultation on the dependent 
measures. TA class means will serve as the unit of analysis. I will use an ANCOVA for 
the experimental portion of the study to assess the group differences in TA behaviors, 
student learning and student motivation. The pretest will serve as the covariate to adjust 
the means of the groups. The analysis will examine overall differences in the 2 
instruments as well as differences in dimension performance such as the attention, 
relevance, confidence and satisfaction dimensions for the CIS. Each instructional activity 
dimension will also be assessed. Additional possible covariates for motivation to be 
examined in the pilot study will include; student desire/likelihood to major in the 
department/field, number of hours (on average) they study/prepare for the course each 
week, or student pretest scores on the CIS. 

Final exam scores will measure learning outcomes. Because I cannot randomly 
assign students to sections, I will attempt to statistically equivalate the groups using the 
following possible covariates for learning: High School GPA, math SAT score, or TA’s 
midterm section average. The pilot study and a further review of the literature may help 
me solidify legitimate covariates for the students. 
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The qualitative findings from the classroom observations, interviews and open- 
ended survey questions on teaching, and utility will be summarized with descriptive 
statistics where possible as well as narrative illustrations. 

Conclusion 

Gagne and Driscoll (1988) state that “Besides the student who is learning, the 
most important agent in the educational program is the teacher. The teacher is 
responsible for arranging the student’s environment to promote learning” (p. 2). 
Hopefully, the student’s and researcher’s feedback related to the systematic instruction 
principles outlined by Reiser and Dick (1996) will enhance the TA’s ability to ‘promote 
learning’ in their classrooms. 
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Appendix B 

*1 Reiser & Dick Instructional Activities Link to Gagne and Learning Theory 



Internal Learning 
Processes 



Gagne's External 
Events of 
Instruction 



Reiser & Dick 
Instructional 
Activities 




c 



9. Enhancing 
retention & transfer 




Motivating students 



j 



Informing students 
of objectives 



Helping students 
recall prerequisites 



Presenting 
information & 
examples 



j 

j 



Providing practice & 
feedback 



j 



i . 

’ Providing enrichment > 

s 

and/or remediation $ 

s 

(optional) ^ 

Vvwwwwwwwwwwww' 



Summarizing the 
lesson 






^Note: Learning process and instructional events from Gagne, (1985). 
ed. 1985. Dashed lines represent a possible, but not complete relationship. 
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Appendix C 

The Effect of Student Ratings of Systematic Instruction 
on Teaching, Learning and Motivation 




Dependent 

Variables 



O 

ERIC 



Proposed Outcomes (Measures) 

+ Increased TA use of Systematic Instruction 
(Feedback Form, Observations, TA Interviews) 

+ Increased Student Learning (Compare Same Final Exam Scores) 
+ Increased Student Motivation (Keller's CIS Motivation Survey) 

+ Assess Utility of R & D Model (Feedback Form, TA Interviews, 
Student Performance/Motivation) 



1 1 1 1 1 1 1 1 1 1 1 1 II 1 1 1 






FLORIDA STATE UNIVERSITY— EVALUATION SERVICES 



GENERAL PURPOSE SURVEY FORM 







MARK ONLY ONE RESPONSE 






PER ITEM. 




- 


.T 


DO NOT STAPLE OR TEAR. 


z. 




^ ( USEANO. 


2J3 EN 1 Cl L ONLY [ _ 






IMPROPER 


PROPER 






MARKS 


MARK 
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Instructional Activities Feedback Form 



I 



Instructions: Your feedback may assist your teacher’s preparation and delivery of future 
classes. Please answer all questions on the space provided using a # 2 pencil. Teachers 
will receive a typed summary of results and NOT see original handwritten comments. 

3 ^ For the following questions, please estimate how often your teacher uses the following 
activities or behaviors during a typical class period. The ratings are: 



E. 



D. 



Almost ALWAYS 



C. 



Often 



Occasionally - 



A. 



L IL Infrequentl y 



Almost NEVER 



A=Almost Never B infrequently C=Occassionally D-Often E-Almost Always 

1 . What is your course name and section #? 



2. What is your teacher’s name? 



3. Begins lesson with an interesting or exciting fact, demonstration, or story related to the class topic. 

4. Actively involves students in discussions, games, or simulations in order to maintain your interest. 

5. Is enthusiastic about the course content and teaching 

6 . Offers praise, rewards, or recognition for correct answers, excellent ideas, or performance 



7. Informs the class of the goals or objectives for each lesson, e.g. By the end of this lesson, you must know the 
following... 

8 . Reviews or refers to prior class topics/discussions that are relevant or necessary to understand the current 
lesson. 

Informs the class of prerequisite skills or knowledge required to understand events in each lesson 9 



9. 



I 



1 1 . Uses several relevant examples to illustrate the lesson topic. 



12. Provides guidance to students to help them remember the key points or aspects of a topic, e.g. Remember the 
colors of the rainbow with the name ‘ROY-G-BIV’ 

13. Provides an opportunity for the student to practice and demonstrate an understanding of the subject 



14. Provides relevant practice problems in a context familiar to the student experiences or knowledge. 

15. Provides feedback related to the practice problems 



16. Provides feedback that guides the students toward the desired outcomes by acknowledging correct answers and 
providing guidance to correct wrong answers. 

17. Summarizes the important aspects of the lesson before dismissing the class 



1 8. Reviews the objectives or goals of the lesson at the end of each lesson. 



19. Provides additional practice or instruction during or after class for students who did not understand the original 
O instruction. 

me Provides enrichment opportunities beyond the normal class activities for students to apply what they learned in 

new and interesting situations. (Continued on back) 

J C 



10. Provides adequate information to understand the key points of each lesson topic 10 
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14 

15 

16 

17 

18 

19 

20 
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j Directions: For the following list of instructional activities in items 22-29, mark column A if you would like 24- 

ii your teacher to do MORE of that activity in class. Mark column B if it is adequately covered in class . 

ij 22. Motivating students to learn 22 

A=do more in class B=adequately covered in class 

23. Informing students of the objectives for the lesson 23 

24. Helping students recall prior knowledge essential for learning the new material 24 

25. Presenting information and relevant examples about the lesson topic 25 

i 26. Providing an opportunity for student practice and teacher feedback on the objectives 2g 

27. Summarizing the lesson 

28. Providing additional practice or remediation opportunities for students 

29. Providing enrichment opportunities for students who desire them 



i 
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30. Overall, how would you rate the usefulness of the activities in questions 22-29 for learning in this course? 

A=Not at all B=Somewhat useful C=Useful D=Very Useful E=Extremely useful 

|jg The following questions allow you to provide more detailed feedback to your instructor and the researcher. Please 
write your answers in the space provided or mark the appropriate letter to the right where appropriate. To ensure 
2 confidentiality, your instructor will be given typed comments and never see the handwritten comments. 

ii 

ii 31. What is the likelihood that you will voluntarily take more classes in this department/field? 31 

if A=Notatall B=Probably not C=Notsure D=Possible E=Definitely 

32. How many hours (on average) do you study/prepare for this course each week? 32 

A= below Vi hour B= 'A to 1 hour C=1 to 1 1 A hours D=1 1 A to 2 hours E= over 2 hours 

i 33. What is your current overall grade point average (GPA) estimate? 33 

A= below 2.0 B= 2.0 -2.5 C= 2.6 -3.0 D= 3.1 -3.5 E= over 3.5 

34. Which category best represents your highest math SAT score? 34 

A= below 400 B= 400 - 460 C= 470 - 530 D= 540 - 600 E= over 600 

35. If you did not take the SAT, which category best represents your highest math ACT score? 35 

A= below 16 B= 16—19 C= 20 - 22 D= 23 -26 E= over 26 

36. Overall, How effective would you rate your teacher’s instruction? 30 

A=Poor B=Below Average C=Average D=Above Average E=Excellent 

37. Describe what you really like about your teacher’s instruction. 37 

38 

39 

40 



38. Describe what you do NOT like about your teacher’s instruction (if any). 
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Appendix E: Consultation Interview Protocol 

1. Name, course number, and section of teacher. 

2. Thank teacher for volunteering to participate in this study and remind them that 
everything discussed is confidential and will not be shared with supervisors, peers etc. 

3. How is the term going? 

4. Give TA chapter 4 of Reiser and Dick text “Planning Instructional Activities.” Show 
teacher results of the student midterm feedback forms (Group mean, individual mean) 
and advise, where appropriate, on integration of the Reiser and Dick instructional 
activities below: 

1 . Motivating students to learn 

2. Informing students of the objectives for the lesson 

3. Helping students recall prior knowledge essential for learning the new material. 

4. Presenting information and relevant examples about the lesson topic. 

5. Providing an opportunity for student practice and teacher feedback on the 
objectives. 

6. Summarizing the lesson 

7. Providing additional practice or remediation opportunities for students. 

8. Providing enrichment opportunities for students who desire them. 

5. Process student handwritten comments from survey. 

6. Show them a model lesson plan that illustrates all these activities 

7. Do you believe that you can incorporate these activities into your classes? If you 
have any questions about implementation during the semester, please don’t hesitate to 
call me and I will gladly discuss it with you. 

(End of term: Was it difficult for you to incorporate these activities?) 

8. How was this feedback useful to you? 

9. Will this feedback help you improve your planning of classroom activities in the 
future? If yes, how will it help your planning? 

10. Do you think this feedback will help you improve your effectiveness as a teacher (e.g. 
students will learn more and be more motivated to learn)? 

1 1 . How would you change the feedback form to make it more relevant to you? 

12. How would you compare this type of feedback to the current SIRS form system? 

13. Do you have any other suggestions to improve this student feedback system? 

14. Thanks again for your time. Remind them of surveys and data required to collect at 
the end of the term. 
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Appendix F: Classroom Observation Checklist (Example) 

Teacher: Subject: Computer Literacy Date/Week/Time:24 Jan/2/2-3 

Lesson Sequence/Topic: 

lminute=admin orientation, brief overview of class with list of topics 
Rest of class: Info presentation & practice/feedback on each task 



Instructional 

Activity 

(Frequency) 


Examples 


Student Response/ 
[Alternative Activity 
(suggested by researcher )] 


Motivating 

Students 

2 


+ Colorful slides w/info and designs- helps gain 
student attention. 

+ Relevance/attention-students had fun drawing 
in paint program to illustrate one of your points 

Alternate activities continued 4 

+ [Learn their names] 

+[Don ’t forget to smile- be enthusiastic when they 
do well, e.g. this is an awesome drawing-future 
Picasso or Monet artist - ^ but NEVER make fun] 


+ Students definitely engaged 
when you gave them the paint 
task. 100% doing task and having 
fun with it. (vs. doing own email, 
other homework etc.) 

+ [Expand on the paint activity to 
see who has the most original 
drawing, most bizarre <£ award 
prizes or recognition. Keeps them 
even more involved, excited] 
+[Use funny, catchy folder names 
to gain attention e.g. 
“Gatormeat'j Also see below 


Inform 
Students of 
Objectives 

Vi 


+ Introduced topic but no criterion mentioned. 
Today we will be discussing windows and email 
(pine), intro to operating systems. 

Alternate activities continued ") 

+ [Would help if agenda and objectives for the 
class were up on the board or screen before class 
so students could see what is expected today. 
Helps activate their stored memory from reading, 
experience etc.] 

+[we will have a quiz at the end of class and you 
will have to successfully demonstrate the 
following to the following standard] 


-Attention really not captured by 
students 

+[Tell them the standards for 
successful completion of each 
topic = criteria] 

+ [Highlight operating system by 
saying (< Todayyou will learn 
where Bill Gates went over a 
billion$ by introducing Windows 
98 and then 2000] 

+ [Highlight use of pine by 
showing a pine tree or bringing in 
a pine tree branch or bunch of 
pine needles-today ’s champion 
gets this as a prize] 


Help Recall 
Prerequisites 

0 


Not done 


+ [link this lesson with the 
previous lesson. How certain 
content you learned from last 
lesson is necessary for this one] 

+ [possibly show diagram of this 
block of unit one in computer 
literacy and where this fits in. e.g. 
here s where we started, where 
we are at now, and where we ' re 
headed next. Building blocks for 
upcoming lessons] 



(continued on back) 
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Classroom Observation Checklist (cont’d) 



Activity 


Examples 


Response///!/*. Activity] 


Present 


+Admin req. for course/# disks, texts etc 


+Students answered question on 


Information & 
Examples 


+DNS topic +PINE 


desktop 


+ Multitasking 


-Many students not following 


+Folders 


along, doing other things(email, 




+OS 


one lady on victoria secret website 




+Difference between 95 & 98 OS 


+[Keep examples as relevant as 




+Composing a message 


possible] 




+Go to address book 


+[When discussing difference 


1=21 


+Defmed hardware/example 


between 95 & 98, could show $$$ 


+Desktop 


and how Bill Gates got more 




H-Right clicking 


money-show newspaper or times 


E=16 


+Open a drive 


magazine picture] 




H-Min/Maximize a window 


+[Walk around to ensure people 




+access control panel 


are staying with you vs. looking at 




+open explore 
+copy file to folder 


websites, doing email etc] 


Practice & 


+log onto pine 


+students were doing the tasks, 


Feedback 


+gave them 3 minutes to do this 


but some not keeping up 


+send email to self 


+several students asked relevant 




+create folder & rename it 


questions for clarification and 




+address book 


were correctly answered 




+Hide and move task bar 


+[have a feedback system where 




+copy file to folder 


you know that class is with you 


P=10 


+If you don’t get it, let me help you 


and ready for next step. E.g. a 


+Feedback given to all students who raised hands 


thumbs up when done, or a little 


F=17 


with a question 


cup or object you place on top of 


+Students practiced at least 10 different tasks 


computer that has a Thumbs up! 




+Quiz at end of class 


for student to put on top of 
computer when finished] 


Summarize 

Lesson 

0 






Enrichment 
Activities (0) 






Remediation 
Activities (0) 






Other 


+Slide introduced each topic->Good transition 


+ [Add criterion for the topic to 


(Combined 
events etc.) 




reinforce what is expected of the 
student for that topic] 
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Course Interest Survey 



I 



Instructions: Please think about each statement in relation to this course and indicate 
how true it is. Give the answer that truly applies to you , and not what you would like 
E to be true, or what you think others want to hear. Think about each statement by itself 



and indicate how true it is. Do not be influenced by your answers to other statements, 
o Please use a # 2 pencil to mark your response. Thank you. The ratings are: 

A=NOT true B=Slightly true C=Moderately true D=Mostly true E-VERY true 



E. 



VERY true 



D, Mostly true 



C. Moderately true 



B. Slightly true 



A. 



NOT true 



I 



1 . What is your course name 



and section #? 



2. What is your teacher’s name? 

3. The instructor knows how to make us feel enthusiastic about the subject matter of this course 3 



4. The things I am learning in this course will be useful to me 4 

5 . I feel confident that I will do well in this course 

6 . This class has very little in it that captures my attention 

7. The instructor makes the subject matter of this course seem important 

8 . You have to be lucky to get good grades in this course 

9. I have to work too hard to succeed in this course 

10. I do NOT see how the content of this course relates to anything I already know 



1 1 . Whether or not I succeed in this course is up to me 

12 

12. The instructor creates suspense when building up to a point 

13. The subject matter of this course is just too difficult for me 13 

14. I feel that this course gives me a lot of satisfaction 14 

15. In this class, I try to set and achieve high standards of excellence 15 

1 6. I feel that the grades or other recognition I receive are fair compared to other students 16 

17. The students in this class seem curious about the subject matter 17 

1 8. I enjoy working for this course 18 



19. I feel satisfied with the instructor’s evaluations of my work compared to how well I think I have done 1 9 



- O 



, am pleased with the instructor’s evaluations of my work compared to how well I think I have done 2 o 

— (Continued on back) 
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21 . 1 feel satisfied with what I am getting from this course 21 

Remember: A=NOT true B=Slightly true C=Moderately true D=Mostlytrue E=VERY true 

22. The content of this course relates to my expectations and goals 22 

23. The instructor does unusual or surprising things that are interesting 23 

24. The students actively participate in this class 24 

25. To accomplish my goals, it is important that I do well in this course 25 



t'A) 



(A) 



26. The instructor uses an interesting variety of teaching techniques 

27. I do NOT think I will benefit much from this course 

28. I often daydream while in class 

29. As I am taking this class, I believe that I can succeed if I try hard enough. 

30. The personal benefits of this course are clear to me 



26 

27 

28 

29 

30 



3 1 . My curiosity is often stimulated by the questions asked or the problems given on the subject matter in this class. 31 

32. I find the challenge level in this course to be about right: neither too easy nor too hard 32 

33. I feel rather disappointed with this course 33 

34. I feel that 1 get enough recognition of my work in this course by means of grades, comments, or other feedback. 34 

35. The amount of work I have to do is appropriate for this type of course 35 

36. 1 get enough feedback to know how well I am doing 35 

37 

38 

39 

40 



Thank you for your honest feedback! 
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D. Mostly true 
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VERY True 
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